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State governments, accrediting agencies, and others are 
demanding that colleges and universities document the progress and 
learning achievements of their students. Within colleges, faculty, 
administrators, and trustee and advisory boards are attempting to 
be more self -regarding and sensitive to student outcomes in 
evaluating and designing courses, programs, and sejrvices. Despite 
this interest, a review of the activity surrounding the 
accountability issue — conferences, journal articles, public 
speeches — finds little mention of the outcomes of continuing 
education courses. Continuing education is growing, both in 
absolute terms and proportionately, at many colleges across the 
country. Yet many campuses are unable to adequately document the 
results of their efforts. It is cur contention that colleges 
should know and be able to report on the student outcomes of their 
noncredit as well as credit programs. 

We begin with a review of the few published, formal 
evaluations of noncredit collegiate programs identified by a search 
of the literature. We will then present a number of reasons for 
conducting a formal assessment of noncredit outcomes, identify 
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obs-taclGs to such assessment, discuss implementation issues, 
recommend several measures of noncredit student outcomes, and 
suggest the creation of several continuing education databases. 

Literature Reviev 

Though many states have issued requirements for assessing 
institutional effectiveness, few have included indicator's of 
noncredit student outcomes in their guidelines o Ohio claims to 
have issued the nation's first statewide standards for noncredit 
continuing education, with the Ohio Board of Regents endorseiaent of 
"Quality Standards for Noncredit Continuing Education Programs" in 
1984 (Ohio Continuing Higher Education Association, 1987) . New 
Jersey included a brief mention of noncredit courses in its listing 
of possible measures of human resource development (Advisory 
Committee to the College Outcomes Evaluation Program, 1987) . 
Several states have included assessment of remedial or 
developmental programs in their accountability guidelines, and 
studies have been completed in this area. For example, the 
California community college system has completed studies of 
student outcomes — goal satisfaction, retention, and skills 
acquisition — in remedial composition and reading (Learning 
Assessment Retention Consortium, 1988) . However, inclusion of 
remedial programs is often within the context of credit program 
assessment. 

The vast majority of the literature on accountability and 
student outcomes assessment focusses exclusively on degree-credit 



programs. Only the rare assessment guidebook or article includes 
any mention of noncredit outcomes. An exception is the model of 
institutional effectiveness developed by the National Alliance of 
Community and Technical Colleges (Grossman and Duncan, 1988), which 
included employment outcomes for noncredit students in its 
indicators of effectiveness in economic development. Also, as this 
was written, the National University Continuing Education 
Association (NUCEA) was circulating "Guidelines for the Assessment 
and Evaluation of Instructional Programs of Continuing Education" 
in draft form (NUCEA, 1990) . 

Even fewer examples of completed studies of noncredit outcomes 
were found. An ERIC search identified some evaluation reports of 
individual programs at individual institutions, but few systematic, 
formal assessments beyond the campus level. A 1986 article in 
Lifelong Learning (Burnham, 1986) suggested that "adult educators 
are sometimes guilty of using convenient numbers that may or may 
not measure what we are really trying to accomplish" and that they 
have "relied upon numbers of people attending programs to 
demonstrate worth, while legislators and other stake holders have 
been asking what differences its activities make in the lives of 
people." The article concluded that we need to move "from inputs 
to impacts" (p. 6). A^ 1990 article in the Continuing Higher 
Education Review (Long, 1990) reported the results of a Delphi 
survey of continuing education deans and directors that concluded 
that "greater attention needs to be paid to learning outcome 
assessment." The ERIC search and a telephone survey of higher 
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education agencies in all 50 states (Diehl, 1990) identified only 
four states in which formal statewide assessments of the learning 
outcomes of noncredit or continuing education had been completed: 
Florida, Kansas^ Maryland, and New York* Community colleges in 
Iowa and Delaware were undertaking such studies in the summer of 
1990, as this was written. 

Florida was the only state identified to date to have included 
noncredit programs in routine, annual program evaluation processes 
at the state level. Noncredit job preparatory programs, but not 
continuing education progrp:3s, are included in the three-level 
yearly program review coordinated by the state. The job prep 
programs have their own annual data displays — one for each program 
at each campus — and are of'^'^en included in the programs identified 
for further review by the Florida community colleges that also 
serve as area vocational centers. Florida's Education and Training 
Placement Information Program, which uses state employer wage 
record files, plus postal service and federal department of defense 
and civil service files to determine the employment (and through 
follow-up surveys, employer satisfaction) of fonnei. vocational 
students, also includes the noncredit job preparatory students. 

The Kansas study (Oaklief , 1987) involved a mail questionnaire 
to test a typology of 6 economic and 13 noneconomic noncredit adult 
learning benefits. Samples were drawn from seven groups of 
noncredit adult learners in Kansas (adult basic education, business 
managers, vocational-technical students, community college 
students, registered nurses, and Pride and non-Pride Cooperative 



Extension Service partici^oants) . The mail survey yielded 1,335 
usable responses. Respondents assigned a value from one (little or 
no benefit) to four (a great benefit) to 19 potential benefits of 
their noncredit learning experience. The top five rated benefits 
across all seven participant groups were (1) becoming better 
informed about some subject, (2) learning recent job knowledge, (3) 
gaining from self improvement, (4) preparing to handle increased 
job responsibility, and (5) improving interest and skill in 
learning. Very little benefits were perceived in improving basic 
reading and writing skills, learning to be effective in politics, 
and increasing appreciation of art and culture. There were some 
differences between groups. Adult basic education participants 
ranked "developing positive feelings about myself and my worth as 
a person" highly. The community college participants ranked 
learning to be effective in politics and increased appreciation of 
a7;t and culture higher than other groups. The overall conclusion 
drawn from the survey was that all seven groups of noncredit 
learners reported both economic and noneconomic benefits from their 
educational experience. 

Lessons learned from the Maryland and New York studies, the 
most comprehensive evaluations of postsecondary continuing 
education completed to date, inform the balance of this essay. 



Reasons for Assessing Continuing Education Outcomes 

There are important reasons why a college, university, 
consortium of institutions, or state system might want to assess 
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continuing education outcomes. Several of these reasons will be 
explored below, with specific references to motivations underlying 
the Maryland community college study. 

To Improve a Growing Area of College operations. The 

fundamental reason for evaluation is to improve the activity 
being evaluated. The need becomes greater as the activity 
increases in importance . At many campuses , noncredit 
continuing education enrollment is growing at a much faster 
rate than credit operations. For example, equated-credit 
fall-time equivalent enrollment in continuing education among 
Maryland's 17 community colleges more than doubled over the 6- 
year period 1983^1989. Even more important than the rate of 
growth, however, is the increasing share of total enrollment 
accounted for by noncredit operations. Noncredit courses 
accounted for nearly a third (31 percent) of all state-funded 
enrollment in Maryland's community colleges in fiscal year 
1989. To omit such a large proportion of college operations 
from formal evaluation efforts would be inexcusable. 

To Establish or Enhance Routine Data Collection Concerning 
Continuing Education. In a March 1989 Policy Statement, the 
Ameri(?an Association of Community and Junior Colleges 
recommended that continuing education professionals "validate 
and disseminate data attesting to the quality of programs and 
services" and that legislators and other policy makers "call 
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separately from students in courses designed for seniors and 
nonvocational students. In New York, the SUNY Office of 
Community Colleges determined that their greatest need was for 
an evaluation of contract courses — courses designed to meet 
the specific needs of clients and delivered according to 
negotiated agreements — and so their initial study focused on 
them (Fadale and Winter, 1988). A second study followed, 
examining remedial, vocational preparation, and community 
services courses. In addition to the quantitative findings, 
the SUNY final report included transcribed comments from 
identified employers attesting to the value of the training 
provided — making the report effective for public relations as 
well as research purposes. 

To Enhanoe the Public Image of Continuing Education. 
Protecting a reputation for quality within the community and 
among public officials may be a concern. Certain recreational 
courses, though self -supported and not subsidized by taxes, 
may undermine the image of continuing education. The issue is 
not whether the courses in question may have intrinsic value 
but whether they might be perceived as frivolous and 
inappropriate offerings for an institution of higher learning. 
While never comprising more than a small fraction of 
continuing education offerings, they may attract a 
disproportionate amount of attention , unduly influencing 
piiblic perceptions of the offering institution. Such concerns 
led to the following recommendation in Blueprint for Quality . 

8 

ER?C 10 



the final report of the Committee o^n the Future of Maryland 
Community Colleges: "That community college continuing 
education divisions limit their noncredit offerings to courses 
that reflect the institution's rcDle in higher education and 
enhance the image of the community college." The committee 
felt that the revenue gains of meeting the demand for these 
recreational offerings mus*: be carefully weighed against 
possible damage done to the reputation of the college. The 
committee was concerned that the public might misunderstand 
which courses were being supported by tax funds, and in 
general that there was little public awareness of what 
continuing education was really about. Better information 
concerning the impact of continuing education courses can 
present a truer picture of continuing education. Measures 
such as student course attendance and completion can 
demonstrate the seriousness of both student and college 
intentions . It is important for these publ ic relations 
purposes that special attention be paid to the presentation of 
study findings to maximize their impact. Presenting data well 
is both art and science, and can be facilitated by adherence 
to proven principles (Clagett and Huntington, 1990) . 



TO Respond Fully to Accountability Mandates. State 
legislatures , higher education agencies , and accrediting 
organizations are requiring student outcomes information in 
various accountability mandates. The size of continuing 

9 

11 



education enrollment at many institutions, especially to the 
extent such courses are tax-assisted, suggests that 
accountability reports should include noncredit student 
outcomes data, even where they are not specifically required* 

To Preserve Tcuc Support for Continuing Education. Tax 

assistance of continuing education varies across the country. 
Where it exists it soon comes to be considered indispensible. 
In Maryland's community college system, the continuing 
education courses which are eligible for state aid are funded 
at the same rate as credit courses. Each college tHerefore 
had a large financial incentive to demonstrate the value of 
these courses to public officials. This was a fundamental, 
though often unmentioned, "bottom-line" reason for the study. 
Similar motivations underlay the SUNY study, and the 
cooperation of individual campuses in the study. The study 
took place during a period of budget discussions at the state 
level when aid for noncredit courses wa5> being threatened 
(Winter and Fadale, 1989). The SUNY report was reviewed by a 
New York state legislative committee while still in draft! 

Common Obstacles to Noncredit Outcomes Assessment 

Accomplishing a useful evaluation of noncredit student 
learning ourcomes is a challenging task. The research professional 
should be aware of and take oteps to overcome the following 
obstacles. 
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Many continuing education administrators are accustomed to 
operating somewhat autonomously on their campuses, and their 
independent, entrepreneurial spirit may resist attempts at formal 
evaluation by "outsiders Continuing education may not, 
therefore, be integrated into the college's routine institutional 
research function- Even simple enrollment tracking and descriptive 
analyses may not have been done before. In states where continuing 
education is not state-assisted, it may not be included in 
accountability mandates and thus the external motivation for 
assessment may be absent. 

Existing databases about the institution's continuing 
education students may be very limited. Often an abbreviated 
course registration form is used for noncredit students, part of 
the overall "ease of entiry" and user-friendliness characteristic of 
the noncredit philosophy. As a result, even basic background 
variables routinely collected and available on the student 
information system for credit students may be lacking for 
continuing education students. 

Many continuing education offerings are of short duration, 
giving the institution little time to greatly influence student 
development. 

Implementation of an Assessment of Continuing Education Outcomes 

Once the decision to conduct a formal evaluation of continuing 
education outcomes has been made, decisions as to approach and 
method arise. The Maryland experience suggests that a study 
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committee comprising both continuing education and research 
professionals can be a fruitful collaboration. The continuing 
education administrators provide expertise on the content of 
noncredit programming and can sensitize the researchers to 
potential areas of resistance to formal evaluation. The 
researchers bring evaluation expertise and can argue the case for 
ongoing assessment and database maintenance. Together, the 
continuing education and research professionals can determine the 
study populations and noncredit student outcomes measures most 
appropriate to the purpose of the study. 

Study Populations 

Students who take continuing education courses are an 
extremely diverse group, and the design of an outcomes assessment 
should take this into account. In order to limit the complexity of 
both the design and reporting of the evaluation, you may wish to 
cover only a portion of the continuing education program. In 
Maryland, the assessment covered only state-supported courses. 
Within that wide variety of courses and students, the research 
design focused on different groups of courses and students. For 
example, students in work-related courses, students in 
nonvocational courses, and senior citizens in special seniors 
courses were analyzed and reported separately. In addition, 
analysis might focus on course intent, with separate evaluation 
mechanisms for open enrollment courses, remedial/ literacy courses, 
courses targeted to students seeking licensure or certification or 
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taking courses as part of their apprenticeship program, and on 
courses designed for particular businesses. The advantage of a 
segmented research design is that assessment instruments can be 
specifically tailored to each study population, resulting in more 
detailed and useful information. Such an approach, however, 
greatly increases the cost, in time, money, and effort. A less 
expensive, less optimal but still useful approach is to employ a 
"generic" evaluation instrument for all students and courses. 
Separate analyses can still be done for specific populations of 
interest, but the questions and responses may not be as sharply 
focused . 

Measures of Noncredit Student Outcomes 

A variety of indicators for measuring noncredit student 
outcomes are available, with their selection depending on the 
nature of the course or program: 

Course attendance. In evaluating credit programs, a basic 
indicator is program completion or at least progress through the 
program. In continuing education courses, which usually don't fit 
together in a structured curriculum, the extent o^ student 
attendance of continuing education course sessions is suggestive of 
the student's seriousness of purpose and is useful in interpreting 
other outcomes findings. This indicator can be gathered from 
instructor records or via a survey question. 
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Completion of course requirements. For courses with specific 
requirements, a measure of completion constitutes a fundamental 
achievement indicator- Awarding of continuing education units or 
CEUs in courses where they are offered is an example. 

Achievement or maintenance of licensure or certification* Real 
estate agents and brokers, property and casualty insurance 
professionals, child care workers, and others may need annual or 
biennial continuing education to maintain valid licenses. 
Instruction to prepare students for the certifying examinations in 
such fields and in other areas such as emergency medicine, food 
sanitation, water safety, pool operation, and waste water 
management are frequently offered in continuing education courses. 
Universities may offer continuing professional education for 
nurses, physicians, and lawyers. A student may be certified, 
licensed, or receive a letter of endorsement by either passing an 
examination or successfully completing required coursework. Pass 
rates on licensure examinations or rates of achievement of 
certification through successful coursework constitute good 
outcomes measures for noncredit instruction. They are clearly 
related to student intent and use externally determined assessment 
criteria. 

Employment • For continuing education programs of sufficient 
duration and purpose, the subsequent employment history of program 
completers may be an appropriate outcomes measure. Such measures 
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are useful in assessing courses in Job Training Partnership Act and 
welfare reform programs, for example. 

Pu2:su5,t of Further Education* The subsequent educational 
experience of course completers may be an appropriate outcome 
measure. The proportion of students taking further noncredit or 
credit courses, and those continuing in an area of interest may, in 
some circumstances, be considered as indicators of favorable 
outcomes . 

Achievement of personal goal. To be meaningful, student outcomes 
assessment must take student goals into account (Clagett, 1989) . 
Students attend college, eid continuing education courses in 
particular, for a wide variety of reasons. Colleges need to know 
why students enroll to better understand their motivations and 
better meet their needs. In addition to job-related goals, such as 
preparing for a new career or updating job skills, students may 
attend for social or enrichment reasons or to explore academic 
options. A study of noncredit outcomes should include 

identification of student goals and the students* perception of the 
extent to which they were met. 

Student satisfaction indices. Measures of student satisfaction 
with their <^-ontinuing education experience, gathered from course 
evaluation forms or subsequent follow-up surveys, provide another 
perspective on program effectiveness. General satisfaction scales, 

15 

ERiC 17 



Databases for Continuing Education Assessment 

Despite the size and growing importance of the cor\tinuing 
education enterprise it is often the area with the poorest 
recordkeeping tools and procedures. Data on students and courses 
may be scattered across multiple computer files, office 
microcomputers, and file cabinets. At some colleges, continuing 
education data management remains a "shoe box" operation. 
Historical data from college files which might be used to track the 
outcomes and progress of students have probably not been collected, 
or may be unevenly coded. In thi? section, we suggest several 
databases that might be designed and maintained to assist in 
assessing noncredit student outcomes. 

Student And Course Information Systems. A continuing education 
student information system which goes beyond accounting information 
may need to be designed and implemented before assessment of 
student outcomes can proceed. Such an information system should 
include many of the data elements routinely collected, stored and 
used to describe the characteristics and goals of students taking 
credit courses, New class registrations procedures may need to be 
established to ensure evenly collected information concerning 
student characteristics such as age, previous educational 
attainment, student goals, and reasons for attending the continuing 
education course . In Maryland , as we built our continuing 
education student database, we also designed a taxonomy for courses 
which classifies each course by subject matter, course intent, and 
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college's course evaluation survey. Two hundred of the 215 
students (93 percent) taking the course were highly satisfied with 
the course content and coverage. Ninety-five percent reported they 
would highly recommend the instructor to their colleagues." 

Follow-up survey files. As mentioned earlier, periodic surveys of 
continuing education students and sponsoring employers can help the 
college understand and explain student goals and expectations and 
now these customers perceived thei*: experience with the college. 
These survey responses can be preserved in data files for future 
analysis, linked tc other databases, and used to maintain address 
files for marketing contacts. 

Licensure and certification examination results > An increasing 
number of continuing education students use the college to prepare 
for employment-related examinations or to fulfill rec[uirements for 
maintaining professional qualifications. In Maryland, we found 
that results from examinations in allied health and real estate 
professions were available by college. However, the records of 
continuing education students in apprenticeship programs and other 
areas requiring external certification did not identify where the 
student had taken his or her preparation courses. In addition to 
establishing their own recordkeeping mechanisms, researchers may 
have to initiate discussions with certifying organizations to 
obtain the desired data- 
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Employment records. Continuing education students taking courses 
as part of a Job Training Partnership Act or similar government- 
sponsored training program often have elaborate work and welfare 
history files. Coordination wi'^h those conducting evaluations of 
these and related programs may provide valuable data regarding both 
the students' experience in courses and their subsequent employment 
history. Several states are exploring processes similar to 
Florida's, ramely the matching of state wage record data with 
college student files to help assess the impact of college on in- 
state employment. 

Conclusions 

As of 1990, fewer than a fifth of the states had undertaken 
formal assessments of postsecondary, noncredit continuing education 
programs. Assessment activities were more frequent in states 
providing funding assistance for noncredit programming, though that 
was no guarantee of formal outcomes study. Many states, however, 
in response to accrediting agency or legislative mandates, have 
begun discussions concerning assessment that would include some 
noncredit offerings. This essay has argued that there are 
important reasons for conducting outcomes assessments in the 
noncredit area, and has suggested several indicators and continuing 
education databases appropriate to such efforts. 
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